TEXT QUALITY

Mention Flags (MF): Constraining Transformer-based Text Generators (2021.acl-long)

Challenge:	Scaling laws in language modeling quantify training loss as a function of dataset size and model parameters, but neglect the critical role of data quality in model generalization.
Approach:	They propose to use effective training tokens as a combination of text diversity and syntheticity as measured by a teacher model to calculate scaling laws.
Outcome:	The proposed term effective training tokens is a combination of two readily-computed indicators of text diversity and syntheticity as measured by a teacher model.

Copied to clipboard

Yufei Wang, Ian Wood, Stephen Wan, Mark Dras, Mark Johnson

Improving Personalized Explanation Generation through Visualization (2022.acl-long)

Challenge:	Constrained decoding algorithms produce hypotheses satisfying all constraints, but they are computationally expensive and can lower the generated text quality.
Approach:	They propose a Mention Flag mechanism which traces whether lexical constraints are satisfied in outputs of an S2S decoder.
Outcome:	The proposed models maintain higher constraint satisfaction and text quality than baseline models and other constrained decoding algorithms.

Copied to clipboard

Shijie Geng, Zuohui Fu, Yingqiang Ge, Lei Li, Gerard de Melo, Yongfeng Zhang

Explainability & Interpretability in NLP

Challenge:	Existing explainable recommendation models generate repetitive sentences for different items or empty sentences with insufficient details.
Approach:	They propose a visual-enhanced approach to generate rating scores and text explanations using visualization generation and text–image matching discrimination.
Outcome:	The proposed approach improves both the text quality and the diversity and explainability of the generated explanations.

Navigating the Path of Writing: Outline-guided Text Generation with Large Language Models (2025.naacl-industry)

Copied to clipboard

Yukyung Lee, Soonwon Ka, Bokyung Son, Pilsung Kang, Jaewook Kang

Improving Cross-Domain Low-Resource Text Generation through LLM Post-Editing: A Programmer-Interpreter Approach (2024.findings-eacl)

Challenge:	Large Language Models (LLMs) have impacted the writing process, enhancing productivity by collaborating with humans in content creation platforms.
Approach:	They propose a framework that uses explicit outlines to guide LLMs in generating goal-oriented, high-quality text.
Outcome:	The proposed approach significantly improves text quality according to evaluations by LLMs and professional writers.

Copied to clipboard

Zhuang Li, Levon Haroutunian, Raj Tumuluri, Philip Cohen, Reza Haf

A CURATEd CATalog: Rethinking the Extraction of Pretraining Corpora for Mid-Resourced Languages (2024.lrec-main)

Challenge:	Large pre-trained language models such as GPT-3.5 and GPT-4 have gained significant attention in natural language research due to limited computational resources or inaccessible parameters.
Approach:	They propose a neural programmer-interpreter approach that preserves the domain generalization ability of LLMs while editing their output.
Outcome:	The proposed framework significantly improves GPT-3.5’s performance in logical form-to-text conversion and low-resource machine translation, surpassing other state-of-the-art (SOTA) LLM post-editing methods in cross-domain settings.

Copied to clipboard

Jorge Palomar-Giner, Jose Javier Saiz, Ferran Espuña, Mario Mina, Severino Da Dalt, Joan Llop, Malte Ostendorff, Pedro Ortiz Suarez, Georg Rehm, Aitor Gonzalez-Agirre, Marta Villegas

So Different Yet So Alike! Constrained Unsupervised Text Style Transfer (2022.acl-long)

Challenge:	CATalog 1.0 is the largest text corpus in Catalan to date . CURATE is a pipeline that can be parallelizable to run in high performance clusters .
Approach:	They propose a data pipeline that uses binary filters to filter documents based on text quality . they optimised the pipeline to run in high performance clusters .
Outcome:	The proposed pipeline is optimized for high performance cluster environments and runs in high performance.

Copied to clipboard

Abhinav Ramesh Kashyap, Devamanyu Hazarika, Min-Yen Kan, Roger Zimmermann, Soujanya Poria

Perceived Text Quality and Readability in Extractive and Abstractive Summaries (2022.lrec-1)

Challenge:	Automated transfer of text between domains does not maintain other attributes between the source and translated text.
Approach:	They propose a method for automatic transfer of text between domains that preserves semantic content but changes other attributes.
Outcome:	The proposed method retains lexical, syntactic and domain-specific constraints between domains for multiple benchmark datasets, including ones where more than one attribute change.

Copied to clipboard

Julius Monsen, Evelina Rennes

Information Extraction & Text Mining

Challenge:	Summarisation systems are limited and reflect human judgements poorly, resulting in expensive and inconsistent evaluation methods.
Approach:	They conducted an online survey on extractive and abstractive summaries using Swedish news data and used them to produce summary.
Outcome:	The summarisation models were trained on Swedish news data and tested on extractive and abstractive summaries.

Eye Movement Features Can Predict Human Preferences on Machine-Generated Texts (2026.acl-srw)

Copied to clipboard

Xiaoshan He, Xiaoqun Liu, Haodong He, Yu Wang, Yang Xu

Anchored Sliding Window: Toward Robust and Imperceptible Linguistic Steganography (2026.acl-long)

Challenge:	Existing studies on eye movement in text quality assessment are limited . eye-movement features are important predictors of human judgments of text quality, but are costly and inconsistent.
Approach:	They propose to capture eye-movement features during screen reading of LLM-generated text using a dataset that includes eye-motion recordings, reading-time measurements, and post-reading evaluations.
Outcome:	The proposed dataset shows that eye-movement features can significantly improve models over other probabilistic metrics, including negative log-likelihood (NLL).

Copied to clipboard

Ruiyi Yan, Shiao Meng, Yugo Murawaki

DOSE: Data Selection for Multi-Modal LLMs via Off-the-Shelf Models (2026.findings-acl)

Challenge:	linguistic steganography assumes that stegographic texts are fragile to even minor modifications, compromising text quality.
Approach:	They propose an anchored sliding window framework to improve imperceptibility and robustness . they propose to include the prompt and a bridge context within the context window .
Outcome:	The proposed framework outperforms the baseline method in text quality, imperceptibility and robustness across diverse settings.

Copied to clipboard

Biao Wu, Yiwu Zhong, Meng Fang, Ling Chen

MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance (D19-1)

Challenge:	Existing data filtering methods are expensive because they are trained on the same data they are meant to screen.
Approach:	They propose to use off-the-shelf pretrained models that have never seen the target data to select training samples for larger and stronger multimodal models without task-specific training.
Outcome:	The proposed method can achieve comparable or even better results than those trained on the full dataset in standard VQA and math benchmarks.

Copied to clipboard

Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Christian M. Meyer, Steffen Eger

Representation Learning

Challenge:	Existing evaluation metrics are not capable of evaluating text quality.
Approach:	They propose a metric that compares system output against reference texts based on semantics rather than surface forms.
Outcome:	The proposed metric shows a high correlation with human judgment of text quality on a number of text generation tasks.

Multi-Task Learning for Coherence Modeling (P19-1)

Copied to clipboard

Youmna Farag, Helen Yannakoudakis

A Cross-Domain Transferable Neural Coherence Model (P19-1)

Challenge:	Existing models for assessing discourse coherence have been developed for summarization and language assessment.
Approach:	They propose a hierarchical neural network that learns to predict a document-level coherence score along with word-level grammatical roles, taking advantage of inductive transfer between the two tasks.
Outcome:	The proposed framework can predict document-level coherence score and word-level grammatical roles using inductive transfer between the two tasks.

Copied to clipboard

Peng Xu, Hamidreza Saghir, Jin Sung Kang, Teng Long, Avishek Joey Bose, Yanshuai Cao, Jackie Chi Kit Cheung

Challenge:	Existing coherence models do not generalize to unseen categories of text . previous work advocates for generative models for cross-domain generalization .
Approach:	They propose a local discriminative neural model with a smaller negative sampling space that can discriminate against incorrect orderings.
Outcome:	The proposed model outperforms state-of-the-art methods on a standard benchmark dataset on the Wall Street Journal corpus and multiple challenging settings on Wikipedia articles.

A Distributional Lens for Multi-Aspect Controllable Text Generation (2022.emnlp-main)

Copied to clipboard

Yuxuan Gu, Xiaocheng Feng, Sicheng Ma, Lingyuan Zhang, Heng Gong, Bing Qin

The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation (2021.emnlp-main)

Challenge:	Existing methods for multi-aspect control suffer from attribute degeneration due to mutual interference of these controllers.
Approach:	They propose to use attribute fusion to find the intersections of multiple attributes as their combination for generation.
Outcome:	The proposed method outperforms baselines on attribute relevance and text quality and achieves the SOTA.

Copied to clipboard

Marzena Karpinska, Nader Akoury, Mohit Iyyer

Heuristic Authorship Obfuscation (P19-1)

Challenge:	Recent research has focused on open-ended text generation tasks because they are difficult to evaluate automatically.
Approach:	They conduct a survey of 45 open-ended text generation papers to determine whether models are reproducible . they then run story evaluation experiments with AMT workers and English teachers .
Outcome:	The results show that AMT workers and English teachers perform better when shown model-generated output alongside human-generated references.

Copied to clipboard

Janek Bevendorff, Martin Potthast, Matthias Hagen, Benno Stein

Leveraging Prefix Transfer for Multi-Intent Text Revision (2023.acl-short)

Challenge:	Existing methods for authorship verification are insufficient to control the authorial style of a text.
Approach:	They propose a novel method that models writing style difference as the Jensen-Shannon distance between character n-gram distributions of texts and manipulates an author’s subconsciously encoded writing style using heuristic search.
Outcome:	The proposed approach defeats state-of-the-art verification approaches while keeping text changes at a minimum.

Copied to clipboard

Ruining Chong, Cunliang Kong, Liu Wu, Zhenghao Liu, Ziye Jin, Liner Yang, Yange Fan, Hanghang Fan, Erhong Yang

Challenge:	Text revision is a necessary process to improve text quality.
Approach:	They propose a multi-intent text revision system that can revise texts without explicit intent annotation.
Outcome:	The proposed system outperforms baselines on the IteraTeR dataset and significantly improves the SARI score with more than 3% improvement.

DiffuVST: Narrating Fictional Scenes with Global-History-Guided Denoising Models (2023.findings-emnlp)

Copied to clipboard

Shengguang Wu, Mei Yuan, Qi Su

ScalingFilter: Assessing Data Quality through Inverse Utilization of Scaling Laws (2024.emnlp-main)

Challenge:	Existing methods for visual storytelling suffer from low inference speed and are not well-suited for synthetic scenes.
Approach:	They propose a diffusion-based system that generates visual descriptions as a single conditional denoising process.
Outcome:	The proposed system improves inter-sentence coherence and image-to-text fidelity.

Copied to clipboard

Ruihang Li, Yixuan Wei, Miaosen Zhang, Nenghai Yu, Han Hu, Houwen Peng

Eyes are the Windows to the Soul: Predicting the Rating of Text Quality Using Gaze Behaviour (P18-1)

Challenge:	Existing quality filtering methods rely on a high-quality dataset as reference . Existing methods introduce potential biases and compromise diversity .
Approach:	They propose a method that evaluates text quality based on the perplexity difference between two language models trained on the same data.
Outcome:	The proposed approach improves performance of pre-trained models without increasing training costs.

Copied to clipboard

Sandeep Mathias, Diptesh Kanojia, Kevin Patel, Samarth Agrawal, Abhijit Mishra, Pushpak Bhattacharyya

Challenge:	Existing methods to predict text quality include estimating subjective aspects of text, like structure, clarity, etc.
Approach:	They propose to capture gaze behaviour to help predict text quality by reporting improvements obtained by adding gaze features to traditional textual features for score prediction.
Outcome:	The proposed model shows that capturing gaze behaviour improves the accuracy of score prediction when the reader has fully understood the text.

Advancing Beyond Identification: Multi-bit Watermark for Large Language Models (2024.naacl-long)

Copied to clipboard

KiYoon Yoo, Wonhyuk Ahn, Nojun Kwak

Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding (2025.findings-naacl)

Challenge:	Existing methods to detect machine-generated text focus on detection, but some misuses require tracing the adversary user for counteracting them.
Approach:	They propose a method for embedding traceable multi-bit information during language model generation.
Outcome:	The proposed method outperforms existing methods in terms of robustness and latency while maintaining text quality.

Copied to clipboard

Kyungmin Min, Minbeom Kim, Kang-il Lee, Dongryeol Lee, Kyomin Jung

Watermark Smoothing Attacks against Language Models (2025.findings-emnlp)

Challenge:	Large Vision-Language Models (LVLMs) generate detailed and coherent responses from visual inputs but are prone to generate hallucinations due to an over-reliance on language priors.
Approach:	They propose a method that reduces the text context and controls only the image-related POS tokens to maintain text quality by reducing the text contextualization.
Outcome:	The proposed method achieves state-of-the-art performance on object hallucination benchmarks and achieves Pareto optimality among the existing methods.

Copied to clipboard

Hongyan Chang, Hamed Hassani, Reza Shokri

Transformer Language Models Handle Word Frequency in Prediction Head (2023.findings-acl)

Challenge:	Watermarking is a key technique for detecting AI-generated text.
Approach:	They propose a method to selectively smooth watermarks by leveraging the relationship between the model’s confidence and detectability.
Outcome:	The proposed method selectively smoothes watermark traces while preserving text quality.

Copied to clipboard

Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, Kentaro Inui

MacLaSa: Multi-Aspect Controllable Text Generation via Efficient Sampling from Compact Latent Space (2023.findings-emnlp)

Challenge:	Prediction head is a crucial component of Transformer language models. Despite its direct impact on prediction, its characteristics have been overlooked in previous analyses.
Approach:	They examine the inner workings of the prediction head, specifically the bias parameters, and quantify the effect of controlling their frequency biases on text generation.
Outcome:	The prediction head is a crucial component of the Transformer language models.

Copied to clipboard

Hanxing Ding, Liang Pang, Zihao Wei, Huawei Shen, Xueqi Cheng, Tat-Seng Chua

Green & Sustainable NLP

Challenge:	Existing approaches to multi-aspect controllable text generation require expensive iteration / searching within the discrete text space during the decoding stage, resulting in a degradation of text quality due to the domain discrepancies between different aspects.
Approach:	They propose a framework that estimates compact latent space for multiple aspects and performs efficient Sampling with a fast sampler to eliminate domain discrepancies.
Outcome:	The proposed framework outperforms baselines on attribute relevance and textual quality while maintaining a high inference speed.

QuantileMark: A Message-Symmetric Multi-bit Watermark for LLMs (2026.acl-long)

Copied to clipboard

Junlin Zhu, Baizhou Huang, Xiaojun Wan

Subtle Signatures, Strong Shields: Advancing Robust and Imperceptible Watermarking in Large Language Models (2024.findings-acl)

Challenge:	a number of large language models (LLMs) require multi-bit watermarking to ensure provenance.
Approach:	They propose a multi-bit watermark that embeds messages within a continuous cumulative probability interval.
Outcome:	The proposed watermark breaks message symmetry in low-entropy decoding, showing it can be used for verification and quality verification.

Copied to clipboard

Yubing Ren, Ping Guo, Yanan Cao, Wei Ma

Evaluating Subtitle Segmentation for End-to-end Generation Systems (2022.lrec-1)

Challenge:	Large Language Models (LLMs) have led to an increase in AI-generated text on the Internet, presenting a crucial challenge to differentiate AI-created content from human-written text.
Approach:	They propose a novel approach to embed watermarks into LLMs that leverages token prior probabilities to improve detectability and maintain watermark imperceptibility.
Outcome:	The proposed method improves detectability and imperceptibility of watermarks by partitioning tokens into two distinct groups based on prior probabilities and employing tailored strategies for each group.

Copied to clipboard

Alina Karakanta, François Buet, Mauro Cettolo, François Yvon

Challenge:	Subtitle segmentation can be evaluated with sequence segmentation metrics against a human reference, but cannot be applied when systems generate outputs different than the reference, e.g. with end-to-end subtitling systems.
Approach:	They propose to use Sigma to evaluate subtitle segmentation against a human reference and a boundary projection method to disentangle the effect of good segmentation from text quality.
Outcome:	The proposed method disentangles the effect of good segmentation from text quality and is compared with existing metrics.

A Reinforcement Learning Framework for Robust and Secure LLM Watermarking (2026.eacl-long)

Copied to clipboard

Li An, Yujian Liu, Yepeng Liu, Yuheng Bu, Yang Zhang, Shiyu Chang

GUARD: Glocal Uncertainty-Aware Robust Decoding for Effective and Efficient Open-Ended Text Generation (2025.findings-emnlp)

Challenge:	Existing watermarking algorithms rely on heuristic green/red token lists . however, these lists are inconsistent and can be compromised .
Approach:	They propose a framework for robust and secure LLM watermarking using reinforcement learning.
Outcome:	The proposed method achieves state-of-the-art trade-off across all criteria with notable improvements in resistance to spoofing attacks without degrading other criteria.

Copied to clipboard

Yuanhao Ding, Esteban Garces Arias, Meimingwei Li, Julian Rodemann, Matthias Aßenmacher, Danlu Chen, Gaojuan Fan, Christian Heumann, Chongsheng Zhang

RepEval: Effective Text Evaluation with LLM Representation (2024.emnlp-main)

Challenge:	GUARD is a self-adaptive decoding method that balances coherence with diversity in open-ended text generation.
Approach:	They propose a self-adaptive decoding method that balances coherence and diversity . they combine global entropy estimates with local entropic deviations to integrate uncertainty .
Outcome:	GUARD achieves a good balance between diversity and coherence while exhibiting significant improvements in generation speed.

Copied to clipboard

Shuqian Sheng, Yi Xu, Tianhang Zhang, Zanwei Shen, Luoyi Fu, Jiaxin Ding, Lei Zhou, Xiaoying Gan, Xinbing Wang, Chenghu Zhou

Representation Learning

Challenge:	Traditional metrics for automatic text evaluation are tailored to specific tasks, while LLM-based evaluation metrics are costly.
Approach:	They propose a metric that leverages projections of LLM representations for evaluation.
Outcome:	The proposed metric exhibits higher correlation with human judgments than previous methods on 14 datasets.

SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models (2026.findings-acl)

Copied to clipboard

Yuhao Wu, Yushi Bai, Zhiqiang Hu, Juanzi Li, Roy Ka-Wei Lee

Generating Diverse Hypotheses for Inductive Reasoning (2025.naacl-long)

Challenge:	Long-form text generation remains a challenge for large language models . generating extended sequences often leads to degraded coherence and logical consistency .
Approach:	They propose a framework that integrates explicit structured thinking into long-form text generation.
Outcome:	The proposed framework surpasses even larger-scale models in evaluation and human evaluation.

Copied to clipboard

Kang-il Lee, Hyukhun Koh, Dongryeol Lee, Seunghyun Yoon, Minsung Kim, Kyomin Jung

Reasoning

Challenge:	Recent studies suggest that large language models (LLMs) can engage in inductive reasoning by sampling multiple hypotheses about the rules and selecting the one that best explains the observations.
Approach:	They propose to increase the temperature parameter to enhance diversity by sampling multiple hypotheses and selecting the one that best explains the observations.
Outcome:	The proposed method improves diversity while maintaining text quality while increasing temperature.

Modeling Structural Similarities between Documents for Coherence Assessment with Graph Convolutional Networks (2023.acl-long)

Copied to clipboard

Wei Liu, Xiyan Fu, Michael Strube

Structured Data in NLP

Challenge:	Existing methods focus on a single document’s coherence patterns, ignoring the underlying correlation between documents.
Approach:	They propose a GCN-based coherence model that captures structural similarities between documents by mining subgraph patterns and a heterogeneous graph for the training corpus.
Outcome:	The proposed model outperforms baseline models on discourse coherence and automated essay scoring tasks.

ORGAN: Observation-Guided Radiology Report Generation via Tree Reasoning (2023.acl-long)

Copied to clipboard

Wenjun Hou, Kaishuai Xu, Yi Cheng, Wenjie Li, Jiang Liu

The Mark Fades: Adaptive Evolutionary Paraphrase-based Attack against LLM Watermarks (2026.findings-acl)

Challenge:	Existing methods to generate radiology reports only rely on high-level plans, but they lack important information.
Approach:	They propose an Observation-guided radiology Report Generation framework which generates free-text descriptions for a set of radiographs.
Outcome:	The proposed framework outperforms state-of-the-art methods regarding text quality and clinical efficacy.

Copied to clipboard

Yusheng Zhao, Jian Zhao, Tianle Zhang, Feng Wei, Xuelong Li

X-Eval: Generalizable Multi-aspect Text Evaluation via Augmented Instruction Tuning with Auxiliary Evaluation Aspects (2024.naacl-long)

Challenge:	Existing paraphrase-based watermark removal methods struggle to balance efficacy with text quality.
Approach:	They propose a training-free evolutionary framework that models watermark removal as a constrained multi-objective optimization problem by using a Pseudo-Log-Likelihood-guided mutation to precisely target and modify watermark-carrying tokens.
Outcome:	The proposed method outperforms baseline methods on a Qwen3 series watermark scheme while maintaining high semantic fidelity.

Copied to clipboard

Minqian Liu, Ying Shen, Zhiyang Xu, Yixin Cao, Eunah Cho, Vaibhav Kumar, Reza Ghanadan, Lifu Huang

Enable Fast Sampling for Seq2Seq Text Diffusion (2024.findings-emnlp)

Challenge:	X-Eval is a two-stage instruction tuning framework to evaluate text in both seen and unseen aspects customized by end users.
Approach:	They introduce a two-stage instruction tuning framework to evaluate text in both seen and unseen aspects customized by end users.
Outcome:	The proposed framework improves the model’s ability to follow evaluation instructions and enhances the learning stage to better assess text quality.

Copied to clipboard

Pan Liu, Xiaohua Tian, Zhouhan Lin

PostMark: A Robust Blackbox Watermark for Large Language Models (2024.emnlp-main)

Challenge:	Existing text generation methods use autoregressive (AR) methods, which generate tokens one by one, but are time-consuming.
Approach:	They propose an efficient model FMSeq which utilizes flow matching to straighten the generation path, thereby enabling fast sampling for diffusion-based seq2seq text generation.
Outcome:	The proposed model generates comparable quality to the SOTA diffusion-based DiffuSeq in just 10 steps, achieving a 200-fold speedup.

Copied to clipboard

Yapei Chang, Kalpesh Krishna, Amir Houmansadr, John Wieting, Mohit Iyyer

From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework for Large Language Models (2025.acl-long)

Challenge:	Existing methods to detect LLM-generated text require access to the underlying LLM’s logits, which LLM providers are loath to share due to fears of model distillation.
Approach:	They develop a post-hoc watermarking procedure that inserts an input-dependent set of words into the text after the decoding process has completed.
Outcome:	The proposed method is more robust to paraphrasing attacks than existing methods.

Copied to clipboard

Yidan Wang, Yubing Ren, Yanan Cao, Binxing Fang

REFFLY: Melody-Constrained Lyrics Editing Model (2025.naacl-long)

Challenge:	Existing watermarking methods face limitations that hinder their effectiveness in diverse and adversarial scenarios.
Approach:	They propose a symbiotic watermarking framework with three strategies: serial, parallel, and hybrid.
Outcome:	The proposed framework outperforms baselines and achieves state-of-the-art (SOTA) performance.

Copied to clipboard

Songyan Zhao, Bingxuan Li, Yufei Tian, Nanyun Peng

User-side Model Consistency Monitoring for Open Source Large Language Models Inference Services (2025.acl-long)

Challenge:	Automatic melody-to-lyric (M2L) generation aims to create lyrics that align with a given melody.
Approach:	They propose a framework for automatic melody-to-lyric generation that allows for a more flexible approach to creating lyrics from plain text.
Outcome:	The proposed framework outperforms baselines Lyra and GPT-4 in musicality and text quality.

Copied to clipboard

Qijun Miao, Zhixuan Fang

SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation (2023.emnlp-main)

Challenge:	Open-source large language models have attracted substantial user base by offering quality comparable to closed-source models at a significantly lower cost.
Approach:	They propose a paradigm for model consistency monitoring on the user side that constructs metrics based on logits produced by LLMs to differentiate sequences generated by degraded models.
Outcome:	The proposed method is implementable on consumer-grade devices and demonstrates that it is cost-effective and reliable.

Copied to clipboard

Elizabeth Clark, Shruti Rijhwani, Sebastian Gehrmann, Joshua Maynez, Roee Aharoni, Vitaly Nikolaev, Thibault Sellam, Aditya Siddhant, Dipanjan Das, Ankur Parikh

A Closer Look into Using Large Language Models for Automatic Evaluation (2023.findings-emnlp)

Challenge:	evaluating the quality of generated text is a difficult problem for large language models.
Approach:	They propose a dataset for multilingual, multifaceted summarization evaluation.
Outcome:	The proposed dataset can be used to train multilingual summarization systems . it shows that the dataset performs well on the out-of-domain meta-evaluation benchmarks TRUE and mFACE .

Copied to clipboard

Cheng-Han Chiang, Hung-yi Lee

XMark: Reliable Multi-Bit Watermarking for LLM-Generated Texts (2026.acl-long)

Challenge:	Existing studies explore the use of large language models to evaluate text quality, but they differ in some details of the evaluation process.
Approach:	They propose to use large language models to evaluate text quality by giving LLMs instructions to evaluate samples by giving them a rating.
Outcome:	The auto Chain-of-Thought (CoT) used in G-Eval does not always make it more aligned with human ratings.

Copied to clipboard

Jiahao Xu, Rui Hu, Olivera Kotevska, Zikai Zhang

Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks (2022.emnlp-main)

Challenge:	Existing methods for embedding binary messages into LLM-generated text suffer from key limitations, such as a poor trade-off between text quality and decoding accuracy.
Approach:	They propose a method for embedding binary messages into Large Language Model (LLM)-generated text that uses a limited number of tokens to decode and recover the encoded message.
Outcome:	The proposed method significantly outperforms existing methods in multiple downstream tasks and will be made publicly available upon acceptance.

Copied to clipboard

Zae Myung Kim, Wanyu Du, Vipul Raheja, Dhruv Kumar, Dongyeop Kang

Syntactic Text Processing

Challenge:	Iterative text revision improves text quality by fixing grammatical errors, rephrasing for better readability or contextual appropriateness.
Approach:	They propose to build an end-to-end text revision system that can iteratively generate helpful edits by explicitly detecting editable spans with their corresponding edit intents.
Outcome:	The proposed system outperforms baselines on other text revision tasks and human evaluations.

Duwak: Dual Watermarks in Large Language Models (2024.findings-acl)

Copied to clipboard

Chaoyi Zhu, Jeroen Galjaard, Pin-Yu Chen, Lydia Chen

Min-k Sampling: Decoupling Truncation from Temperature Scaling via Relative Logit Dynamics (2026.acl-long)

Challenge:	Existing watermark techniques are effective in embedding single human-imperceptible and machine-detectable patterns without significantly affecting generated text quality and semantics.
Approach:	They propose to embed dual secret patterns in token probability distribution and sampling schemes to enhance the efficiency of watermarking.
Outcome:	The proposed method achieves highest watermark quality at the lowest required token count for detection, up to 70% less than existing techniques, especially under post paraphrasing attacks.

Copied to clipboard

Yuanhao Ding, Meimingwei Li, Esteban Garces Arias, Matthias Aßenmacher, Christian Heumann, Chongsheng Zhang

Challenge:	Existing methods for decoding large language models have extreme sensitivity to temperature parameter T.
Approach:	They propose a dynamic truncation strategy that analyzes the local shape of the logit distribution to identify "semantic cliffs" they show that Min-k consistently improves text quality even under extreme temperature settings .
Outcome:	The proposed method achieves strict temperature invariance and low sensitivity to hyperparameter choices.

Best-k Search Algorithm for Neural Text Generation (2023.acl-long)

Copied to clipboard

Jiacheng Xu, Caiming Xiong, Silvio Savarese, Yingbo Zhou

Green & Sustainable NLP

Challenge:	Modern natural language generation paradigms require a decoding strategy to obtain quality sequences out of the model.
Approach:	They propose a deterministic search algorithm balancing quality and diversity . they investigate the vanilla best-first search algorithm and propose k-k search algorithm.
Outcome:	The proposed algorithm is parameter-free, lightweight, efficient, and easy-to-use.

LLMs as Narcissistic Evaluators: When Ego Inflates Evaluation Scores (2024.findings-acl)

Copied to clipboard

Yiqi Liu, Nafise Moosavi, Chenghua Lin

An Extensible Plug-and-Play Method for Multi-Aspect Controllable Text Generation (2023.acl-long)

Challenge:	Existing evaluation metrics for natural language generation tasks favor text generated by different LMs . human evaluation by experts is the most reliable approach, but it is costly and time-consuming .
Approach:	They examine whether language model-driven evaluation metrics exhibit bias toward underlying language models in the context of summarization tasks.
Outcome:	The proposed evaluation metrics tend to assign inflated scores to outputs generated by the very model they are based on.

Copied to clipboard

Xuancheng Huang, Zijun Liu, Peng Li, Tao Li, Maosong Sun, Yang Liu

RAFT: Realistic Attacks to Fool Text Detectors (2024.emnlp-main)

Challenge:	Multi-aspect controllable text generation has attracted increasing attention . but the mutual interference of multiple prefixes limits its extensibility to training-time unseen combinations.
Approach:	They propose to use trainable gates to normalize the intervention of prefixes to restrain the interference.
Outcome:	The proposed approach outperforms baselines on constraint accuracy, text quality, and extensibility.

Copied to clipboard

James Wang, Ran Li, Junfeng Yang, Chengzhi Mao

Prompt-based Generation of Natural Language Explanations of Synthetic Lethality for Cancer Drug Discovery (2024.lrec-main)

Challenge:	Large language models (LLMs) have exhibited remarkable fluency across tasks, but their unethical applications are unclear.
Approach:	They propose a grammar error-free black-box attack that exploits LLM embeddings at the word-level while preserving original text quality.
Outcome:	The proposed attack compromises all detectors across domains and is transferable across source models.

Copied to clipboard

Ke Zhang, Yimiao Feng, Jie Zheng

DualGuard: Dual-stream Large Language Model Watermarking Defense against Paraphrase and Spoofing Attack (2026.findings-acl)

Challenge:	Synthetic lethality (SL) is a genetic interaction where a single gene mutation allows cell survival, but simultaneous mutations in two genes lead to cell death.
Approach:	They propose a prompt-based pipeline for generating natural language explanations using a dataset derived from New Bing .
Outcome:	The proposed pipeline improves on existing biomedical language models in terms of text quality and explainability.

Copied to clipboard

Hao Li, Yubing Ren, Yanan Cao, Yingjie Li, Fang Fang, Shi Wang, Li Guo

Topic-Based Watermarks for Large Language Models (2026.findings-acl)

Challenge:	Existing watermarking algorithms focus on defending against paraphrase and piggyback spoofing attacks, which can inject harmful content, compromise reliability, and undermine trust in attribution.
Approach:	They propose an algorithm capable of defending against paraphrase and spoofing attacks.
Outcome:	Experiments on large language models and language models show that DualGuard is the first watermarking algorithm capable of defending against both paraphrase and spoofing attacks.

Copied to clipboard

Alexander Nemecek, Yuzhou Jiang, Erman Ayday

Perspective-driven Preference Optimization with Entropy Maximization for Diverse Argument Generation (2025.findings-emnlp)

Challenge:	Existing watermarking methods often involve trade-offs between attack robustness, generation quality and additional overhead.
Approach:	They propose a topic-guided watermarking scheme that partitions the vocabulary into topic-aligned token subsets.
Outcome:	The proposed method achieves text quality comparable to industry-leading systems and improves watermark robustness against paraphrasing and lexical perturbation attacks with minimal performance overhead.

Copied to clipboard

Yilin Cao, Ruike Zhang, Penghui Wei, Qingchao Kong, Wenji Mao

Reasoning

Challenge:	Argument generation with diverse perspectives is essential for fostering balanced discourse and mitigating bias.
Approach:	They propose a Perspective-aware Preference Optimization with Entropy Maximization framework for diverse argument generation.
Outcome:	The proposed framework enhances perspective diversity through preference optimization based on the constructed preference dataset .

Holistic Evaluation for Interleaved Text-and-Image Generation (2024.emnlp-main)

Copied to clipboard

Minqian Liu, Zhiyang Xu, Zihao Lin, Trevor Ashby, Joy Rimchala, Jiaxin Zhang, Lifu Huang

Improving LLM-as-a-Judge Inference with the Judgment Distribution (2025.findings-emnlp)

Challenge:	Existing evaluation benchmarks do not support arbitrarily interleaved images and text for both inputs and outputs.
Approach:	They propose to use a benchmark to evaluate interleaved text-and-image generation . they define five evaluation aspects for InterleavatedEval, a reference-free metric .
Outcome:	The proposed benchmarks cover a limited number of domains and use cases and lack comparableity-based metrics.

Copied to clipboard

Victor Wang, Michael JQ Zhang, Eunsol Choi

Challenge:	Using language models to scalably approximate human preferences on text quality (LLM-as-a-judge) is a standard practice applicable to many tasks.
Approach:	They propose to use LLM judges to approximate human preferences on text quality by using distributional output over judgment tokens.
Outcome:	The proposed method outperforms taking the mode (i.e. greedy decoding) in all evaluation settings, and incorporating risk aversion improves performance.

Context-aware Watermark with Semantic Balanced Green-red Lists for Large Language Models (2024.emnlp-main)

Copied to clipboard

Yuxuan Guo, Zhiliang Tian, Yiping Song, Tianlun Liu, Liang Ding, Dongsheng Li

Filtered Direct Preference Optimization (2024.emnlp-main)

Challenge:	Recent research suggests that watermarking methods cause degradation of text quality due to semantic disparities between the watermarked text and the unwatermarked text.
Approach:	They propose a semantic-aware watermark method that generates a watermark key considering contexts to split a green/red list for watermark injection.
Outcome:	The proposed method reduces performance drop due to adding bias on green lists . it also allows green lists to cover almost all semantics .

Copied to clipboard

Tetsuro Morimura, Mitsuki Sakamoto, Yuu Jinnai, Kenshi Abe, Kaito Ariu

Challenge:	Existing studies on the impact of RLHF on text quality have focused on reward-model-free RL.
Approach:	They propose an extension of direct preference optimization to improve model performance by analyzing the quality of the preference dataset.
Outcome:	The proposed method improves the performance of models optimized with DPO over those optimized with reward-model-based RLHF.

Measuring What Matters Beyond Text: Evaluating Multimodal Summaries by Quality, Alignment, and Diversity (2026.findings-acl)

Copied to clipboard

Abid Ali, Diego Molla, Usman Naseem

SimMark: A Robust Sentence-Level Similarity-Based Watermarking Algorithm for Large Language Models (2025.emnlp-main)

Challenge:	MLLMs have facilitated multimodal summarization with multimodal outputs, but their evaluation is fragmented . MM-Eval integrates assessments of textual quality, cross-modal alignment, and visual diversity .
Approach:	They propose a unified evaluation framework that integrates assessments of textual quality, cross-modal alignment, and visual diversity.
Outcome:	The proposed framework improves over heuristic aggregation baselines and provides an interpretable, reference-weak framework for comparative evaluation of multimodal summaries.

Copied to clipboard

Amirhossein Dabiriaghdam, Lele Wang

TempParaphraser: “Heating Up” Text to Evade AI-Text Detection through Paraphrasing (2025.emnlp-main)

Challenge:	Experimental results demonstrate that SimMark surpasses previous sentence-level watermarking techniques in robustness, sampling efficiency, and applicability across diverse domains.
Approach:	They propose a sentence-level watermarking algorithm that makes LLM outputs traceable without requiring access to model internals.
Outcome:	The proposed algorithm surpasses previous sentence-level watermarking techniques in robustness, sampling efficiency, and applicability across diverse domains while maintaining the text quality and fluency.

Copied to clipboard

Junjie Huang, Ruiquan Zhang, Jinsong Su, Yidong Chen